A Generalized Gittins Index for a Markov Chain and its Recursive Calculation
نویسنده
چکیده
We discuss a generalization of the classical Gittins Index for a Markov chain and propose a transparent recursive algorithm for its calculation. The foundation for this algorithm is a modified version of the Elimination algorithm proposed earlier by the author to solve the problem of optimal stopping of a Markov chain in discrete time and a finite or countable state space.
منابع مشابه
Optimal Stopping of Markov Chain and Three Abstract Optimization Problems
There is a well known connection between three problems related to Optimal Stopping of Markov Chain and the equality of three corresponding indices: the classical Gittins index in the Ratio Maximization Problem, the Kathehakis-Veinot index in a Restart Problem, and Whittle index in a family of Retirement Problems. In [13] these three problems and these three indices were generalized in such a w...
متن کاملRestart Probability Model
We discuss a new applied probability model: there is a system whose evolution is described by a Markov chain (MC) with known transition matrix on a discrete state space and at each moment of a discrete time a decision maker can apply one of three possible actions: continue, quit, and restart MC in one of a finite number of fixed “restarting” points. Such a model is a generalization of a model d...
متن کاملA (2/3)n3 Fast-Pivoting Algorithm for the Gittins Index and Optimal Stopping of a Markov Chain
T paper presents a new fast-pivoting algorithm that computes the n Gittins index values of an n-state bandit—in the discounted and undiscounted cases—by performing 2/3 n3 +O n2 arithmetic operations, thus attaining better complexity than previous algorithms and matching that of solving a corresponding linearequation system by Gaussian elimination. The algorithm further applies to the problem of...
متن کاملOne-armed bandit models with continuous and delayed responses
One-armed bandit processes with continuous delayed responses are formulated as controlled stochastic processes following the Bayesian approach. It is shown that under some regularity conditions, a Gittins-like index exists which is the limit of a monotonic sequence of break-even values characterizing optimal initial selections of arms for finite horizon bandit processes. Furthermore, there is a...
متن کاملMapping Activity Diagram to Petri Net: Application of Markov Theory for Analyzing Non-Functional Parameters
The quality of an architectural design of a software system has a great influence on achieving non-functional requirements of a system. A regular software development project is often influenced by non-functional factors such as the customers' expectations about the performance and reliability of the software as well as the reduction of underlying risks. The evaluation of non-functional paramet...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2005